Rethinking attention mechanism in time series classification

The potential of attention mechanisms in time series classification (TSC) is limited owing to general drawbacks, like weak local perception and quadratic complexity. To promote the performance mechanisms, we present a flexible multi-head linear (FMLA) architecture, which enhances locality awareness through layer-wise interactions with deformable convolutional blocks online knowledge distillation. We develop simple but effective mask mechanism that helps reduce noise influence reduces redundancy FMLA by probabilistically selecting masking positions each given series. use incremental ablation studies on 85 UCR2018 datasets evaluate main techniques developed. Experimental results demonstrate outperformed 11 state-of-the-art TSC algorithms, obtaining mean accuracy 89.37%. achieved best 29 short-medium 7 long time-series regarding accuracy. has complexity ON, where N sample length. As increases from 100 1000, floating-point operations per second grow linearly 0.13G 1.34G.